Articulated arms are controllable machines that can accurately and repeatably move an end effector throughout a workspace to a specified pose. As used herein, the term pose illustratively refers to a combination of three dimensional position and three dimensional rotation. A robot pose illustratively corresponds to the pose of the end effector in terms of the robot's coordinate system. Illustratively, end effectors are controllable machines that are capable of grasping and releasing workpieces and may comprise, e.g., mechanical grippers, vacuum suction, electromagnetic contacts, etc. Generally, as used herein, the combination of an articulated arm and an end effector is termed a robot. The three dimensional area that is accessible by the robot for performing operations is termed a workplace for the robot. Robots are illustratively utilized to perform tasks such as moving a workpiece within the workspace by performing a predetermined sequence of steps. For example, the end effector may be moved along trajectory T1 to end effector pose P1. The end effector may then grasp a workpiece before moving along trajectory T2 to end effector pose P2. Once at pose P2, the end effector may then release the workpiece. A known disadvantage of utilizing such robots is that the workpiece must be presented at a known pose in order for the robot to successfully perform the task. For example, the end effector may fail to grasp the workpiece if the workpiece is not initially located at the expected pose. Workpiece pose inaccuracy is a well-known noted disadvantage of such robot systems and is a typical reason that robots are not selected for use in performing pick, place, and assembly operations.
One known technique to eliminate workpiece pose inaccuracy is to utilize sensors to first measure the workpiece pose and then to utilize that measured workpiece pose to adjust the robot's trajectories. Machine vision is a typical technique utilized to sense workpiece positions as machine vision is typically a fast, inexpensive, non-contact sensing modality. The term Vision Guided Robotics (VGR) illustratively refers to the process of using a machine vision system to help the robot perform tasks.
A noted disadvantage of using machine vision to measure a workpiece's pose is that the machine vision system typically measures the workpiece pose with respect to the machine vision system's coordinate system, whereas the robot moves the end effector with respect to the robot's coordinate system. Thus, the workpiece pose computed by the machine vision system must be translated into the robot's coordinate system in order for the robot to make use of the workpiece pose computed by the machine vision system. The term hand-eye calibration illustratively refers to the task of determining the relationship between the machine vision system's coordinate system and the robot's coordinate system.
As will be appreciated by those skilled in the art, the accuracy of the hand-eye calibration directly affects the accuracy of the VGR procedures. If the hand-eye calibration accurately reflects the true relationship between the machine vision system's coordinate system and the robot's coordinate system, then accurate machine vision measurements of the workpiece pose will induce accurate end effector poses for picking, placing, and assembling the workpiece. Correspondingly, if the hand-eye calibration is inaccurate and does not reflect the true relationship between the machine vision system's coordinate system and the robot's coordinate system, then accurate machine vision measurement of the workpiece will not necessarily induce accurate end effector poses for picking, placing, and assembling the workpiece. These inaccurate robot poses may result in the robot failing to perform the intended tasks.
Tsai and Lenz developed a well-known technique for performing hand-eye calibration. Their technique is described in “A new technique for fully autonomous and efficient 3D robotics hand/eye calibration”, IEEE Transactions on Robotics Automation, Volume 5, Issue 3, pages 345-348, the contents of which are hereby incorporated by reference. Tsai and Lenz's technique illustratively obtains a set of robot poses and associated acquired images. For each of the acquired images, the technique estimates the pose of the camera with respect to the calibration object. This produces a set of robot poses and estimated poses of the camera with respect to the calibration object. Tsai and Lenz's technique then determines the pose of the camera with respect to the articulated arm from the set of robot poses and corresponding estimated camera poses. In an exemplary case of a stationary camera environment, a noted disadvantage of Tsai and Lenz's technique is that it does not explicitly incorporate the constraint that there is one and only one pose of the camera with respect to the robot base and that there is one and only one pose of the calibration object with respect to the end effector. Instead, Tsai and Lenz's technique allows for independent, therefore possibly inconsistent, estimates of the camera poses and further allows for independent, therefore possibly inconsistent, estimates of the calibration object poses.
A further noted disadvantage of Tsai and Lenz's technique is that it only performs calibration for a single camera in a machine vision system at a time. Thus, if it is desired to perform calibration for a plurality of cameras in a machine vision system, the Tsai and Lenz technique must be performed repeatedly for each of the cameras. This increases the time required to perform calibration and allows for independent and therefore possibly inconsistent calibration for individual cameras.
Tsai and Lenz considered pairwise motions and considered the expression AiX=XBi where Ai characterizes the perceived motion of the camera and Bi characterizes a relative motion of the end effector. Thereby, they could compute the transform between the end effector and the camera X by solving for the X which minimized the overall discrepancies between pairs of AXi and XBi. In “Optimal Hand-Eye Calibration”, published in the Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008 pages 4647-4653, Strobl and Hirzinger mentioned a hand-eye calibration technique which considered the expression AX=ZB to estimate the end-effector-camera transform and the robot-calibration plate transforms, the problem is reduced to a system of rigid transforms: AiX=ZBi, where Ai characterizes the perceived transform between the camera and the calibration plate, X characterizes the end effector-camera transform, Z characterizes the robot base-calibration plate transform, and Bi characterizes the end effector pose (the transform from the end effector to the robot base). The drawback of both of these approaches (and all other approaches following the basic technique) is that they involve estimating differences between two rigid transforms: diff(Transform1,Transforms2), and there is no well-understood, physically-meaningful way of measuring the difference between two transforms.
Another drawback of methods like those described by Tsai & Lens and Strobl and Hirzinger is that estimating the hand-eye calibration based only on calibration plate poses is suboptimal. This is because each computed camera-calibration plate pose is usually more precise in some directions than other directions, but that directional precision information is not contained in the camera-calibration plate poses. The fact that camera-calibration plate poses are usually more precise in some directions than other directions can be understood by considering that the calibration plate's orientation can usually be more precisely estimated than the calibration plate's tilt.